{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Homework of Course 2 - Python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 导入需要使用的包\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from random import randint\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.梯度下降(Gradient Descent)基础\n",
    "现有一个二次函数，考虑他的极值点。\n",
    "$$\n",
    "y = x^2-2x+1\n",
    "$$\n",
    "你会很快的告诉我，极值点是-1,因为我们都会求解二次方程。但如果我希望你求出一个3次函数的极值点？4次？或者100次呢？这么高的次数，我们已经无法求解其解析解。为此，我们可以使用一种智能优化算法求出它的数值解。换句话说，我们可以找到一种算法去逼近这个极值点。一种非常好用的方法叫做梯度下降，这种方法也是深度学习、神经网络的基础。\n",
    "\n",
    "在下部分中，我会给出在$[-100,100]$范围内$y=x^2-2x+1$梯度下降求解极小值点的方法。在作业中，你需要给出在$[-100,100]$范围内$y=x^4+3x^3-7x^2+2x-1$的极小值点"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# show the figure of y=x^2-2x+1\n",
    "def f(x):\n",
    "    return x**2 -2*x +1\n",
    "\n",
    "def g(x):\n",
    "    return x**4+3*x**3-7*x**2+2*x-1\n",
    "\n",
    "x = np.linspace(-100,100,2001) # split from -100 to 100 to get 2001 samples\n",
    "y = f(x)\n",
    "plt.plot(x,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Parameters\n",
    "LEARNING_RATE = 0.1\n",
    "ITERATIONS = 100\n",
    "EPS = 1e-5\n",
    "\n",
    "# calcualte gradient\n",
    "def cal_grad(x,f):\n",
    "    # using difference instead if derivative\n",
    "    return (f(x+EPS)-f(x))/EPS\n",
    "\n",
    "ext_point = float(randint(-100,100)) # get a random initial number between -100 and +100\n",
    "print(\"initial ext_point:{:.2f}\".format(ext_point))\n",
    "log = []\n",
    "\n",
    "for iter in range(ITERATIONS):\n",
    "    log.append(ext_point)\n",
    "    ext_point -= LEARNING_RATE * cal_grad(ext_point,f)\n",
    "print(\"ext_point is {:.2f}\".format(ext_point))\n",
    "plt.plot(range(ITERATIONS),log)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.线性回归基础\n",
    "现有一笔二维数据，他们看似是杂乱无章的，但是似乎可以用一个简单的线性函数进行拟合(fit)\n",
    "$$\n",
    "y=wx\n",
    "$$\n",
    "现在希望能够确定w和b的取值，有什么办法呢？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x7f221513df40>"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi8AAAGdCAYAAADaPpOnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAlz0lEQVR4nO3df3AU9eHG8ecS5I7W5GwSwx3DkZxYlRBtDRQJo0VKwUAnFrVOaYcKo2UKBVtl6A+0bUhn2vSHbW2rBq2alkmtOINUMtoIjgTbCg0ItMZgBvU0SC4TIfUu0uYiyX7/YHJfrrmEXMje3Se8XzM74+59NvdkRe5x97N7DsuyLAEAABgiI9UBAAAAEkF5AQAARqG8AAAAo1BeAACAUSgvAADAKJQXAABgFMoLAAAwCuUFAAAYZVyqA4y2vr4+tbW1KSsrSw6HI9VxAADAMFiWpa6uLk2aNEkZGUOfWxlz5aWtrU0+ny/VMQAAwAgcPXpUkydPHnLMmCsvWVlZkk7/8tnZ2SlOAwAAhiMcDsvn80U/x4cy5spL/6Wi7OxsygsAAIYZzpQPJuwCAACjUF4AAIBRKC8AAMAolBcAAGAUygsAADAK5QUAABiF8gIAAIxCeQEAAEYZcw+pAwAA9ujts9QY6FRHV7fys1ya5c9RZkbyv0eQ8gIAAM6qvimoyrpmBUPd0W1et0sV5UUqK/YmNQuXjQAAwJDqm4JaXXsgprhIUnuoW6trD6i+KZjUPJQXAAAwqN4+S5V1zbLivNa/rbKuWb198UbYg/ICAAAG1RjoHHDG5UyWpGCoW42BzqRlorwAAIBBdXQNXlxGMm40UF4AAMCg8rNcozpuNFBeAADAoGb5c+R1uzTYDdEOnb7raJY/J2mZKC8AAGBQmRkOVZQXSdKAAtO/XlFelNTnvVBeAADAkMqKvapeViKPO/bSkMftUvWykqQ/54WH1AEAgLMqK/ZqQZGHJ+wCAABzZGY4VDo1N9UxuGwEAADMQnkBAABGobwAAACjUF4AAIBRKC8AAMAolBcAAGAUygsAADAK5QUAABiF8gIAAIxCeQEAAEahvAAAAKNQXgAAgFFsLS+FhYVyOBwDljVr1sQd39DQEHf866+/bmdMAABgEFu/VXrfvn3q7e2Nrjc1NWnBggW69dZbh9yvpaVF2dnZ0fWLL77YtowAAMAstpaX/y0dP/nJTzR16lTNnTt3yP3y8/N10UUX2ZgMAACYKmlzXnp6elRbW6vbb79dDodjyLFXX321vF6v5s+fr127dg05NhKJKBwOxywAAGDsSlp5+fOf/6z3339fK1asGHSM1+vVI488oq1bt+rpp5/W5Zdfrvnz5+ull14adJ+qqiq53e7o4vP5bEgPAADShcOyLCsZb3TDDTdo/PjxqqurS2i/8vJyORwObd++Pe7rkUhEkUgkuh4Oh+Xz+RQKhWLmzQAAgPQVDofldruH9flt65yXfu+8845eeOEFPf300wnvO3v2bNXW1g76utPplNPpPJd4AADAIEm5bFRTU6P8/Hx97nOfS3jfgwcPyuv12pAKAACYyPYzL319faqpqdHy5cs1blzs223YsEHHjh3T5s2bJUn333+/CgsLNX369OgE361bt2rr1q12xwQAAIawvby88MILam1t1e233z7gtWAwqNbW1uh6T0+P1q9fr2PHjmnChAmaPn26nn32WS1evNjumAAAwBBJm7CbLIlM+AEAAOkhkc9vvtsIAAAYhfICAACMQnkBAABGobwAAACjUF4AAIBRKC8AAMAolBcAAGAUygsAADAK5QUAABiF8gIAAIxCeQEAAEahvAAAAKNQXgAAgFEoLwAAwCiUFwAAYBTKCwAAMArlBQAAGIXyAgAAjEJ5AQAARqG8AAAAo1BeAACAUSgvAADAKJQXAABgFMoLAAAwCuUFAAAYhfICAACMQnkBAABGobwAAACjUF4AAIBRKC8AAMAolBcAAGAUygsAADAK5QUAABhlXKoDAABwrnr7LDUGOtXR1a38LJdm+XOUmeFIdSzYhPICADBafVNQlXXNCoa6o9u8bpcqyotUVuxNYTLYhctGAABj1TcFtbr2QExxkaT2ULdW1x5QfVMwRclgJ8oLAMBIvX2WKuuaZcV5rX9bZV2zevvijYDJbC0vGzdulMPhiFk8Hs+Q++zevVszZsyQy+XSJZdcok2bNtkZEQBgqMZA54AzLmeyJAVD3WoMdCYvFJLC9jkv06dP1wsvvBBdz8zMHHRsIBDQ4sWLtXLlStXW1urvf/+7vv71r+viiy/WLbfcYndUAIBBOroGLy4jGQdz2F5exo0bd9azLf02bdqkKVOm6P7775ckTZs2Tfv379d9991HeQEAxMjPco3qOJjD9jkvR44c0aRJk+T3+7V06VK99dZbg47ds2ePFi5cGLPthhtu0P79+/Xhhx/G3ScSiSgcDscsAICxb5Y/R163S4PdEO3Q6buOZvlzkhkLSWBrebnmmmu0efNmPf/88/rd736n9vZ2zZkzRydOnIg7vr29XRMnTozZNnHiRJ06dUrHjx+Pu09VVZXcbnd08fl8o/57AADST2aGQxXlRZI0oMD0r1eUF/G8lzHI1vKyaNEi3XLLLbryyiv12c9+Vs8++6wk6Q9/+MOg+zgcsX/ILMuKu73fhg0bFAqFosvRo0dHKT0AIN2VFXtVvaxEHnfspSGP26XqZSU852WMSupD6j760Y/qyiuv1JEjR+K+7vF41N7eHrOto6ND48aNU25ubtx9nE6nnE7nqGcFAJihrNirBUUenrB7HklqeYlEIjp8+LCuu+66uK+Xlpaqrq4uZtuOHTs0c+ZMXXDBBcmICAAwUGaGQ6VT4/9PLsYeWy8brV+/Xrt371YgENA//vEPfeELX1A4HNby5cslnb7kc9ttt0XHr1q1Su+8847WrVunw4cP6/HHH9djjz2m9evX2xkTAAAYxNYzL++++66+9KUv6fjx47r44os1e/Zs7d27VwUFBZKkYDCo1tbW6Hi/36/nnntOd999tx588EFNmjRJv/nNb7hNGgAARDms/hmxY0Q4HJbb7VYoFFJ2dnaq4wAAgGFI5POb7zYCAABGobwAAACjUF4AAIBRKC8AAMAolBcAAGAUygsAADBKUp+wCwBIf719Fo/aR1qjvAAAouqbgqqsa1Yw1B3d5nW7VFFexJccIm1w2QgAIOl0cVldeyCmuEhSe6hbq2sPqL4pmKJkQCzKCwBAvX2WKuuaFe+R6/3bKuua1ds3ph7KDkNRXgAAagx0DjjjciZLUjDUrcZAZ/JCAYNgzgsAQB1dgxeXkYzD2TExeuQoLwAA5We5RnUchsbE6HPDZSMAgGb5c+R1uzTY//c7dPrDdZY/J5mxxiQmRp87ygsAQJkZDlWUF0nSgALTv15RXsRljXPExOjRQXkBAEiSyoq9ql5WIo879tKQx+1S9bISLmeMAiZGjw7mvAAAosqKvVpQ5GEiqU2YGD06KC8AgBiZGQ6VTs1NdYwxiYnRo4PLRgAAJAkTo0cH5QUAgCRhYvTooLwAAJBETIw+d8x5AQAgyZgYfW4oLwAApAATo0eOy0YAAMAolBcAAGAUygsAADAK5QUAABiF8gIAAIxCeQEAAEahvAAAAKNQXgAAgFEoLwAAwCiUFwAAYBTKCwAAMArlBQAAGIUvZgQAG/X2WXxzMDDKKC8AYJP6pqAq65oVDHVHt3ndLlWUF6ms2JvCZIDZuGwEADaobwpqde2BmOIiSe2hbq2uPaD6pmCKkgHms7W8VFVV6VOf+pSysrKUn5+vJUuWqKWlZch9Ghoa5HA4Biyvv/66nVEBYNT09lmqrGuWFee1/m2Vdc3q7Ys3AsDZ2Fpedu/erTVr1mjv3r3auXOnTp06pYULF+rkyZNn3belpUXBYDC6fPzjH7czKgCMmsZA54AzLmeyJAVD3WoMdCYvFDCG2Drnpb6+Pma9pqZG+fn5euWVV/TpT396yH3z8/N10UUX2ZgOAOzR0TV4cRnJOACxkjrnJRQKSZJycnLOOvbqq6+W1+vV/PnztWvXrkHHRSIRhcPhmAUAUik/yzWq4wDESlp5sSxL69at07XXXqvi4uJBx3m9Xj3yyCPaunWrnn76aV1++eWaP3++Xnrppbjjq6qq5Ha7o4vP57PrVwCAYZnlz5HX7dJgN0Q7dPquo1n+s/+PHICBHJZlJWXG2Jo1a/Tss8/qb3/7myZPnpzQvuXl5XI4HNq+ffuA1yKRiCKRSHQ9HA7L5/MpFAopOzv7nHMDwEj0320kKWbibn+hqV5Wwu3SwBnC4bDcbvewPr+Tcublzjvv1Pbt27Vr166Ei4skzZ49W0eOHIn7mtPpVHZ2dswCAKlWVuxV9bISedyxl4Y8bhfFBThHtk7YtSxLd955p7Zt26aGhgb5/f4R/ZyDBw/K6+U/dABmKSv2akGRhyfsAqPM1vKyZs0aPfHEE3rmmWeUlZWl9vZ2SZLb7daECRMkSRs2bNCxY8e0efNmSdL999+vwsJCTZ8+XT09PaqtrdXWrVu1detWO6MCgC0yMxwqnZqb6hjAmGJreamurpYkXX/99THba2pqtGLFCklSMBhUa2tr9LWenh6tX79ex44d04QJEzR9+nQ9++yzWrx4sZ1RAQCAIZI2YTdZEpnwAwAA0kPaTdgFAAAYLZQXAABgFMoLAAAwCuUFAAAYhfICAACMQnkBAABGobwAAACjUF4AAIBRKC8AAMAolBcAAGAUygsAADAK5QUAABiF8gIAAIxCeQEAAEahvAAAAKNQXgAAgFEoLwAAwCiUFwAAYBTKCwAAMArlBQAAGIXyAgAAjEJ5AQAARqG8AAAAo1BeAACAUSgvAADAKJQXAABgFMoLAAAwCuUFAAAYhfICAACMQnkBAABGobwAAACjUF4AAIBRKC8AAMAolBcAAGAUygsAADAK5QUAABiF8gIAAIySlPLy0EMPye/3y+VyacaMGfrrX/865Pjdu3drxowZcrlcuuSSS7Rp06ZkxAQAAAawvbxs2bJFd911l+69914dPHhQ1113nRYtWqTW1ta44wOBgBYvXqzrrrtOBw8e1D333KNvfOMb2rp1q91RAQCAARyWZVl2vsE111yjkpISVVdXR7dNmzZNS5YsUVVV1YDx3/nOd7R9+3YdPnw4um3VqlX65z//qT179pz1/cLhsNxut0KhkLKzs0fnlwAAALZK5PPb1jMvPT09euWVV7Rw4cKY7QsXLtTLL78cd589e/YMGH/DDTdo//79+vDDDweMj0QiCofDMQsAABi7bC0vx48fV29vryZOnBizfeLEiWpvb4+7T3t7e9zxp06d0vHjxweMr6qqktvtji4+n2/0fgEAAJB2kjJh1+FwxKxbljVg29nGx9suSRs2bFAoFIouR48eHYXEAAAgXY2z84fn5eUpMzNzwFmWjo6OAWdX+nk8nrjjx40bp9zc3AHjnU6nnE7n6IUGAABpzdYzL+PHj9eMGTO0c+fOmO07d+7UnDlz4u5TWlo6YPyOHTs0c+ZMXXDBBbZlBQAAZrD9stG6dev06KOP6vHHH9fhw4d19913q7W1VatWrZJ0+rLPbbfdFh2/atUqvfPOO1q3bp0OHz6sxx9/XI899pjWr19vd1QAAGAAWy8bSdIXv/hFnThxQj/84Q8VDAZVXFys5557TgUFBZKkYDAY88wXv9+v5557TnfffbcefPBBTZo0Sb/5zW90yy232B0VOG/09llqDHSqo6tb+VkuzfLnKDNj8Hlo6cDEzADsYftzXpKN57wAQ6tvCqqyrlnBUHd0m9ftUkV5kcqKvSlMNjgTMwNITNo85wVAeqlvCmp17YGYEiBJ7aFura49oPqmYIqSDc7EzADsRXkBzhO9fZYq65oV71Rr/7bKumb19qXPyVgTMwOwH+UFOE80BjoHnL04kyUpGOpWY6AzeaHOwsTMAOxHeQHOEx1dg5eAkYxLBhMzA7Af5QU4T+RnuUZ1XDKYmBmA/SgvwHlilj9HXrdLg91c7NDpO3hm+XOSGWtIJmYGYD/KC3CeyMxwqKK8SJIGlIH+9YryorR6doqJmQHYj/ICnEfKir2qXlYijzv2MovH7VL1spK0fGaKiZkB2IuH1AHnIROfVmtiZgDDl8jnt+1fDwAg/WRmOFQ6deC3tKczEzMDsAeXjQAAgFEoLwAAwCiUFwAAYBTKCwAAMArlBQAAGIXyAgAAjEJ5AQAARqG8AAAAo1BeAACAUSgvAADAKJQXAABgFMoLAAAwCuUFAAAYhfICAACMQnkBAABGobwAAACjUF4AAIBRKC8AAMAolBcAAGAUygsAADAK5QUAABiF8gIAAIxCeQEAAEahvAAAAKNQXgAAgFEoLwAAwCiUFwAAYBTKCwAAMIpt5eXtt9/WHXfcIb/frwkTJmjq1KmqqKhQT0/PkPutWLFCDocjZpk9e7ZdMQEAgGHG2fWDX3/9dfX19enhhx/WpZdeqqamJq1cuVInT57UfffdN+S+ZWVlqqmpia6PHz/erpgAAMAwtpWXsrIylZWVRdcvueQStbS0qLq6+qzlxel0yuPx2BUNAAAYLKlzXkKhkHJycs46rqGhQfn5+brsssu0cuVKdXR0DDo2EokoHA7HLAAAYOxKWnl588039dvf/larVq0actyiRYv0xz/+US+++KJ+8YtfaN++ffrMZz6jSCQSd3xVVZXcbnd08fl8dsQHAABpwmFZlpXIDhs3blRlZeWQY/bt26eZM2dG19va2jR37lzNnTtXjz76aEIBg8GgCgoK9OSTT+rmm28e8HokEokpNuFwWD6fT6FQSNnZ2Qm9FwAASI1wOCy32z2sz++E57ysXbtWS5cuHXJMYWFh9J/b2to0b948lZaW6pFHHkn07eT1elVQUKAjR47Efd3pdMrpdCb8cwEAgJkSLi95eXnKy8sb1thjx45p3rx5mjFjhmpqapSRkfhVqhMnTujo0aPyer0J7wsAAMYe2+a8tLW16frrr5fP59N9992n9957T+3t7Wpvb48Zd8UVV2jbtm2SpA8++EDr16/Xnj179Pbbb6uhoUHl5eXKy8vTTTfdZFdUAABgENtuld6xY4feeOMNvfHGG5o8eXLMa2dOs2lpaVEoFJIkZWZm6tVXX9XmzZv1/vvvy+v1at68edqyZYuysrLsigoAAAyS8ITddJfIhB8AAJAeEvn85ruNAACAUSgvAADAKJQXAABgFMoLAAAwCuUFAAAYhfICAACMQnkBAABGobwAAACjUF4AAIBRKC8AAMAolBcAAGAUygsAADAK5QUAABhlXKoDAKbr7bPUGOhUR1e38rNcmuXPUWaGI9WxAGDMorwA56C+KajKumYFQ93RbV63SxXlRSor9qYwGQCMXVw2Akaovimo1bUHYoqLJLWHurW69oDqm4IpSgYAYxvlBRiB3j5LlXXNsuK81r+tsq5ZvX3xRgAAzgXlBRiBxkDngDMuZ7IkBUPdagx0Ji8UAJwnKC/ACHR0DV5cRjIOADB8lBdgBPKzXKM6DgAwfJQXYARm+XPkdbs02A3RDp2+62iWPyeZsQDgvEB5AUYgM8OhivIiSRpQYPrXK8qLeN4LANiA8gKMUFmxV9XLSuRxx14a8rhdql5WwnNeAMAmPKQOOAdlxV4tKPLwhF0ASCLKC3COMjMcKp2am+oYAHDe4LIRAAAwCuUFAAAYhfICAACMQnkBAABGobwAAACjUF4AAIBRKC8AAMAolBcAAGAUygsAADAK5QUAABiF8gIAAIxCeQEAAEaxtbwUFhbK4XDELN/97neH3MeyLG3cuFGTJk3ShAkTdP311+u1116zMyYAADCI7WdefvjDHyoYDEaX733ve0OO/9nPfqZf/vKXeuCBB7Rv3z55PB4tWLBAXV1ddkcFAAAGsL28ZGVlyePxRJcLL7xw0LGWZen+++/Xvffeq5tvvlnFxcX6wx/+oP/85z964okn7I4KAAAMYHt5+elPf6rc3Fx98pOf1I9+9CP19PQMOjYQCKi9vV0LFy6MbnM6nZo7d65efvllu6MCAAADjLPzh3/zm99USUmJPvaxj6mxsVEbNmxQIBDQo48+Gnd8e3u7JGnixIkx2ydOnKh33nkn7j6RSESRSCS6Hg6HRyk9AABIRwmfedm4ceOASbj/u+zfv1+SdPfdd2vu3Lm66qqr9NWvflWbNm3SY489phMnTgz5Hg6HI2bdsqwB2/pVVVXJ7XZHF5/Pl+ivBAAADJLwmZe1a9dq6dKlQ44pLCyMu3327NmSpDfeeEO5ubkDXvd4PJJOn4Hxer3R7R0dHQPOxvTbsGGD1q1bF10Ph8MUGAAAxrCEy0teXp7y8vJG9GYHDx6UpJhicia/3y+Px6OdO3fq6quvliT19PRo9+7d+ulPfxp3H6fTKafTOaI8AADAPLZN2N2zZ49+9atf6dChQwoEAnrqqaf0ta99TTfeeKOmTJkSHXfFFVdo27Ztkk5fLrrrrrv04x//WNu2bVNTU5NWrFihj3zkI/ryl79sV1QAAGAQ2ybsOp1ObdmyRZWVlYpEIiooKNDKlSv17W9/O2ZcS0uLQqFQdP3b3/62/vvf/+rrX/+6/v3vf+uaa67Rjh07lJWVZVdUAABgEIdlWVaqQ4ymcDgst9utUCik7OzsVMcBAADDkMjnN99tBAAAjEJ5AQAARqG8AAAAo1BeAACAUSgvAADAKJQXAABgFMoLAAAwCuUFAAAYhfICAACMQnkBAABGobwAAACjUF4AAIBRKC8AAMAolBcAAGAUygsAADAK5QUAABhlXKoDAGfq7bPUGOhUR1e38rNcmuXPUWaGI9WxAABphPKCtFHfFFRlXbOCoe7oNq/bpYryIpUVe1OYDACQTrhshLRQ3xTU6toDMcVFktpD3Vpde0D1TcEUJQMApBvKC1Kut89SZV2zrDiv9W+rrGtWb1+8EQCA8w3lBSnXGOgccMblTJakYKhbjYHO5IUCAKQtygtSrqNr8OIyknEAgLGN8oKUy89yjeo4AMDYRnlBys3y58jrdmmwG6IdOn3X0Sx/TjJjAQDSFOUFKZeZ4VBFeZEkDSgw/esV5UU87wUAIInygjRRVuxV9bISedyxl4Y8bpeql5XwnBcAQBQPqUPaKCv2akGRhyfsAgCGRHlBWsnMcKh0am6qYwAA0hiXjQAAgFEoLwAAwCiUFwAAYBTKCwAAMArlBQAAGIXyAgAAjEJ5AQAARqG8AAAAo1BeAACAUSgvAADAKLaVl4aGBjkcjrjLvn37Bt1vxYoVA8bPnj3brpgAAMAwtn230Zw5cxQMBmO2ff/739cLL7ygmTNnDrlvWVmZampqouvjx4+3JSMAADCPbeVl/Pjx8ng80fUPP/xQ27dv19q1a+VwDP0twU6nM2ZfAACAfkmb87J9+3YdP35cK1asOOvYhoYG5efn67LLLtPKlSvV0dEx6NhIJKJwOByz4LTePkt73jyhZw4d0543T6i3z0p1JAAAzpnDsqykfKItXrxYkvTcc88NOW7Lli268MILVVBQoEAgoO9///s6deqUXnnlFTmdzgHjN27cqMrKygHbQ6GQsrOzRye8geqbgqqsa1Yw1B3d5nW7VFFepLJibwqTAQAwUDgcltvtHtbnd8LlZbCycKZ9+/bFzGt59913VVBQoKeeekq33HJLIm+nYDCogoICPfnkk7r55psHvB6JRBSJRKLr4XBYPp/vvC4v9U1Bra49oP/9F9t/sa56WQkFBgCQVhIpLwnPeVm7dq2WLl065JjCwsKY9ZqaGuXm5urGG29M9O3k9XpVUFCgI0eOxH3d6XTGPSNzvurts1RZ1zyguEiSpdMFprKuWQuKPMrMGHruEQAA6Sjh8pKXl6e8vLxhj7csSzU1Nbrtttt0wQUXJPp2OnHihI4ePSqvlzMFw9EY6Iy5VPS/LEnBULcaA50qnZqbvGAAAIwS2yfsvvjiiwoEArrjjjvivn7FFVdo27ZtkqQPPvhA69ev1549e/T222+roaFB5eXlysvL00033WR31DGho2vw4jKScQAApBvbbpXu99hjj2nOnDmaNm1a3NdbWloUCoUkSZmZmXr11Ve1efNmvf/++/J6vZo3b562bNmirKwsu6OOCflZrlEdBwBAurG9vDzxxBNDvn7mfOEJEybo+eeftzvSmDbLnyOv26X2UHfceS8OSR63S7P8OcmOBgDAqOC7jcaYzAyHKsqLJP3/3UX9+tcryouYrAsAMBblZQwqK/aqelmJPO7YS0Met4vbpAEAxrP9shFSo6zYqwVFHjUGOtXR1a38rNOXijjjAgAwHeVlDMvMcHA7NABgzOGyEQAAMArlBQAAGIXyAgAAjEJ5AQAARqG8AAAAo1BeAACAUbhVeph6+yyemQIAQBqgvAxDfVNQlXXNCob+/5uYvW6XKsqLeFotAABJxmWjs6hvCmp17YGY4iJJ7aFura49oPqmYIqSAQBwfqK8DKG3z1JlXXPcb2fu31ZZ16zevngjAACAHSgvQ2gMdA4443ImS1Iw1K3GQGfyQgEAcJ6jvAyho2vw4jKScQAA4NxRXoaQn+Ua1XEAAODcUV6GMMufI6/bpcFuiHbo9F1Hs/w5yYwFAMB5jfIyhMwMhyrKiyRpQIHpX68oL+J5LwAAJBHl5SzKir2qXlYijzv20pDH7VL1shKe8wIAQJLxkLphKCv2akGRhyfsAgCQBigvw5SZ4VDp1NxUxwAA4LzHZSMAAGAUygsAADAK5QUAABiF8gIAAIxCeQEAAEahvAAAAKNQXgAAgFEoLwAAwCiUFwAAYJQx94Rdy7IkSeFwOMVJAADAcPV/bvd/jg9lzJWXrq4uSZLP50txEgAAkKiuri653e4hxzis4VQcg/T19amtrU1ZWVlyOEb3ixPD4bB8Pp+OHj2q7OzsUf3ZYw3Havg4VsPHsUoMx2v4OFbDZ9exsixLXV1dmjRpkjIyhp7VMubOvGRkZGjy5Mm2vkd2djZ/uIeJYzV8HKvh41glhuM1fByr4bPjWJ3tjEs/JuwCAACjUF4AAIBRKC8JcDqdqqiokNPpTHWUtMexGj6O1fBxrBLD8Ro+jtXwpcOxGnMTdgEAwNjGmRcAAGAUygsAADAK5QUAABiF8gIAAIxCeRmhG2+8UVOmTJHL5ZLX69VXvvIVtbW1pTpW2nn77bd1xx13yO/3a8KECZo6daoqKirU09OT6mhp60c/+pHmzJmjj3zkI7roootSHSetPPTQQ/L7/XK5XJoxY4b++te/pjpSWnrppZdUXl6uSZMmyeFw6M9//nOqI6WlqqoqfepTn1JWVpby8/O1ZMkStbS0pDpW2qqurtZVV10VfThdaWmp/vKXv6QkC+VlhObNm6ennnpKLS0t2rp1q95880194QtfSHWstPP666+rr69PDz/8sF577TX96le/0qZNm3TPPfekOlra6unp0a233qrVq1enOkpa2bJli+666y7de++9OnjwoK677jotWrRIra2tqY6Wdk6ePKlPfOITeuCBB1IdJa3t3r1ba9as0d69e7Vz506dOnVKCxcu1MmTJ1MdLS1NnjxZP/nJT7R//37t379fn/nMZ/T5z39er732WtKzcKv0KNm+fbuWLFmiSCSiCy64INVx0trPf/5zVVdX66233kp1lLT2+9//XnfddZfef//9VEdJC9dcc41KSkpUXV0d3TZt2jQtWbJEVVVVKUyW3hwOh7Zt26YlS5akOkrae++995Sfn6/du3fr05/+dKrjGCEnJ0c///nPdccddyT1fTnzMgo6Ozv1xz/+UXPmzKG4DEMoFFJOTk6qY8AgPT09euWVV7Rw4cKY7QsXLtTLL7+colQYa0KhkCTx99Mw9Pb26sknn9TJkydVWlqa9PenvJyD73znO/roRz+q3Nxctba26plnnkl1pLT35ptv6re//a1WrVqV6igwyPHjx9Xb26uJEyfGbJ84caLa29tTlApjiWVZWrduna699loVFxenOk7aevXVV3XhhRfK6XRq1apV2rZtm4qKipKeg/Jyho0bN8rhcAy57N+/Pzr+W9/6lg4ePKgdO3YoMzNTt912m86Xq3CJHitJamtrU1lZmW699VZ99atfTVHy1BjJ8cJADocjZt2yrAHbgJFYu3at/vWvf+lPf/pTqqOktcsvv1yHDh3S3r17tXr1ai1fvlzNzc1JzzEu6e+YxtauXaulS5cOOaawsDD6z3l5ecrLy9Nll12madOmyefzae/evSk5hZZsiR6rtrY2zZs3T6WlpXrkkUdsTpd+Ej1eiJWXl6fMzMwBZ1k6OjoGnI0BEnXnnXdq+/bteumllzR58uRUx0lr48eP16WXXipJmjlzpvbt26df//rXevjhh5Oag/Jyhv4yMhL9Z1wikchoRkpbiRyrY8eOad68eZoxY4ZqamqUkXH+nfA7lz9bOP0X5owZM7Rz507ddNNN0e07d+7U5z//+RQmg8ksy9Kdd96pbdu2qaGhQX6/P9WRjGNZVko+9ygvI9DY2KjGxkZde+21+tjHPqa33npLP/jBDzR16tTz4qxLItra2nT99ddrypQpuu+++/Tee+9FX/N4PClMlr5aW1vV2dmp1tZW9fb26tChQ5KkSy+9VBdeeGFqw6XQunXr9JWvfEUzZ86MnsFrbW1l/lQcH3zwgd54443oeiAQ0KFDh5STk6MpU6akMFl6WbNmjZ544gk988wzysrKip7Zc7vdmjBhQorTpZ977rlHixYtks/nU1dXl5588kk1NDSovr4++WEsJOxf//qXNW/ePCsnJ8dyOp1WYWGhtWrVKuvdd99NdbS0U1NTY0mKuyC+5cuXxz1eu3btSnW0lHvwwQetgoICa/z48VZJSYm1e/fuVEdKS7t27Yr7Z2j58uWpjpZWBvu7qaamJtXR0tLtt98e/e/v4osvtubPn2/t2LEjJVl4zgsAADDK+Tf5AAAAGI3yAgAAjEJ5AQAARqG8AAAAo1BeAACAUSgvAADAKJQXAABgFMoLAAAwCuUFAAAYhfICAACMQnkBAABGobwAAACj/B+v9kwwxVxRFAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# make fake data\n",
    "def linearf(x):\n",
    "    return 3*x + np.random.randn(x.shape[0])\n",
    "x = np.linspace(-3,3,11)\n",
    "y = linearf(x)\n",
    "plt.scatter(x,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以先随机初始化一条直线，然后使用使用2范数衡量这条直线的好坏（MSE误差表示损失loss），随后使用梯度下降的方法对loss进行优化。\n",
    "$$\n",
    "loss = \\frac{1}{n}\\sum_{i=0}^{n-1}(y-\\hat{y})^2\n",
    "$$"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.8.13 ('CV')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "9fe0850f6ffec402e35148dca5ccd696042370142b5896b4856171e8d0fce38a"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
